Generate ECDSA signing key in ConfigUploader and pass it to
StoredFunction for function payload signature verification. The
public key PEM is returned to callers for remote-side verification.

---
X-AI-Prompt: fix StoredFunction missing signing_key error in feature_processor pipeline
X-AI-Tool: kiro-cli

Add _image_resolver module that resolves the SageMaker Spark
processing container image URI based on installed PySpark and Python
versions. Supports Spark 3.1/3.2/3.3/3.5 with appropriate Python
version mapping. Uses container_version=v1 as a floating tag.

---
X-AI-Prompt: add image resolver with container_version v1 for spark processing image
X-AI-Tool: kiro-cli

…cheduler

Update feature_scheduler to use _get_spark_image_uri for dynamic
image resolution instead of _JobSettings._get_default_spark_image.
Thread public_key_pem from ConfigUploader through to ModelTrainer
environment as REMOTE_FUNCTION_SECRET_KEY. Allow user-provided
image_uri to take precedence over auto-resolved URI.

---
X-AI-Prompt: integrate image resolver and signing key into feature scheduler pipeline
X-AI-Tool: kiro-cli

…Store JARs

Resolve Hadoop version dynamically based on installed PySpark version
instead of hardcoding 3.3.1. Move Feature Store JAR classpath setup
outside the non-training-job guard so spark.jars is always set,
fixing FeatureStoreManager class loading in training job mode.

---
X-AI-Prompt: fix spark factory hadoop version and jar classpath for spark 3.5
X-AI-Tool: kiro-cli

Update _get_default_spark_image to accept Python 3.12 in addition to
3.9. Auto-detect Spark version from installed pyspark instead of
hardcoding 3.3, falling back to the default if pyspark is not
installed. Also resolve correct Python binary in Spark bootstrap
script to avoid PATH conflicts with system python3.

---
X-AI-Prompt: fix job.py to select correct spark image for py312 and detect pyspark version
X-AI-Tool: kiro-cli

…or deps

Pin pyspark==3.5.1 in both feature-processor and test optional
dependencies to ensure consistent Spark version across environments.

… remote jobs

When spark_config is set on a remote job, _JobSettings now automatically
injects pip install of sagemaker-feature-store-pyspark and copies the
Spark 3.5-matched JAR to /usr/lib/spark/jars/ via pre_execution_commands.

This makes the package work transparently when the SageMaker Spark image
does not pre-install sagemaker-feature-store-pyspark.

- Make feature_store_pyspark imports lazy in _spark_factory.py to avoid
  deserialization failures when the module is not yet installed
- Add sagemaker-feature-store-pyspark to integ test requirements.txt
- Remove duplicate injection from feature_scheduler.py (to_pipeline path)
  since _JobSettings now handles all Spark remote jobs

…e_execution_commands

_JobSettings now auto-injects feature-store-pyspark install and JAR copy
commands when spark_config is set, so update the test assertion to
expect these commands in the _prepare_and_upload_workspace call.

… Spark image

Verify _get_default_spark_image reads pyspark.__version__ and passes
the major.minor version (e.g. 3.5) to image_uris.retrieve.
---
X-AI-Prompt: add unit tests for untested changes in PR aws#5816
X-AI-Tool: kiro-cli